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Abstract 

The Gauss-Newton's method for solving nonlinear least squares problems is studied in this 
paper. Under the hypothesis that the derivative of the function associated with the least square 
problem satisfies a majorant condition, a local convergence analysis is presented. This analysis 
allow us to obtain the optimal convergence radius, the biggest range for the uniqueness of 
solution, and to unify two previous and unrelated results. 

Keywords: Nonlinear least squares problems; Gauss-Newton's method; Majorant condition; Local 
convergence. 

1 Introduction 

The Gauss-Newton's method is one of the most efficient methods known for solving nonlinear least 
squares problems 

min ^F(x) T F(x), (1) 

where F : f2 — > R m is differentiable function, f2 C M ra is an open set and m > n. Formally, the 
Gauss- Newton's method is described as follows: Given a initial point xq £ £1, define 

x k+1 = x k - [F'(x k ) T F'(x k )] -1 F'(x k ) T F(x k ), k = 0, 1, ... . 
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The convergence of this method may fail or it even fail to be well defined. To ensure that the method 
is well defined and converges to a solution of (pQ), some conditions must be impose. For instance, 
the classical convergence analysis (see [H [2] ) requires that F' satisfies the Lipschitz condition and 
the initial iterate to be "close enough" the solution, but it cannot make us clearly see how big is 
the convergence radius of the ball. 

In the last years, there are many papers dealing with the convergence of the Newton's methods, 
including the Gauss-Newton's method, by relaxing the assumption of Lipschitz continuity of the 
derivative (see [3 HI El El [H El [TUl [IH [121 HH H]). Those works in addition to improving the 
convergence theory (this allows us estimate the convergence radius and to enlarge the range of 
application) also permit us unify two results. 

Our aim in this paper is present a new local convergence analysis for Gauss-Newton's method 
under major ant condition introduced by Kantorovich |15j . and used with successful by Ferreira 
and Goncalves [5], Ferreira [6] and Ferreira and Svaiter [7] for studying Newton's method. In our 
analysis, the classical Lipschitz condition is relaxed using a majorant function. It is worth pointing 
out that this condition is equivalent to Wang's condition introduced in [9] and used by Chen, Li 
[3JU] and Li, et al. [8] for studying Gauss- Newton and Newton's method. The convergence analysis 
presented provides a clear relationship between the majorant function, which relax the Lipschitz 
continuity of the derivate, and the function associated with the nonlinear least square problem, 
see for example Lemmas [T3| [T4l and [I5j Thus, the results presented here has the conditions and 
proof of convergence simpler and more didactic. Also, as in Chen, Li [3], it allow us to obtain the 
biggest range for the uniquess of solution and the optimal convergence radius for the method with 
respect to majorant function. Moreover, two unrelated previous results pertaining Gauss-Newton's 
method are unified. 

The organization of the paper is as follows. In Sect. 11.14 we list some notations and basic 
results used in our presentation. In Sect. [2] the main result is stated, and in Sect. 12.11 some prop- 
erties involving the majorant function are established. In Sect. 12.21 we presented the relationships 
between the majorant function and the non-linear function F, and in Sect. 12.31 the optimal ball of 
convergence and the uniqueness of solution of convergence are established. In Sect. 12.41 the main 
result is proved and some applications of this result are given in Sect. [3j 

1.1 Notation and auxiliary results 

The following notations and results are used throughout our presentation. The open and closed 
ball at a £ R n and radius 5 > are denoted, respectively by 

B(a, 5) = {x £ R n ; \\x - a\\ < 5}, B[a, 5] = {x e R n ; \\x - a\\ < 6}. 

Let R mXTl denote the set of all m x n matrix A, denote the Moore-Penrose inverse of matrix A, 
and if A has full rank(namely: rank(A)=min(m,n)=n) then A< = (A T A)~ l A T . 

Lemma 1. (Banach's Lemma) Let B G ]g mxm and I G M. mxm , the identity operator. If\\B—I\\ < 1, 
then B is invertible and \\B~ 1 \\ < 1/(1 — \\B — I\\) . 
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Proof. See the proof of Lemma 1, p. 189 of Smale |16j with A = I and c = \\B — □ 

Lemma 2. Suppose that A, E £ R mxn (m > n), B = A + E, \\EA^\\ < 1, rank(A)=n, then 
rank(B)=n. 

Proof. In fact, B = A + E = (I + EA^)A, from the condition ||-EM.t|| < 1, we have of Lemma U 
that I + EA^ is invertivel. So rank(B)=rank(A)=n. □ 



Lemma 3. Suppose that A,E G R mxn , B = A + E, P T ||||£|| < 1, rank(A) = rank(B), then 

11 11 - i-\m\\EW 

Moreover, if rank(A) = rank(B) = min{m,n), there holds 

11 "-1-L4* E 



Proof. See Lema 5.1. on pp. 40 of Stewart |17| and Wedin [18]. □ 
Proposition 4. If < t < 1, then £°^ (i + 2 )(^ + = 2 /( 1 ~ f ) 3 - 

Proof. Take k = 2 in Lemma 3, pp. 161 of Blum, et al. [19]. □ 

Also, the following auxiliary results of elementary convex analysis will be needed: 
Proposition 5. Let R > 0. If <p : [0, R) — )■ M is convex, then 

D^(p(0) = hm u ^o+ = mf <« • 

u u 

Proof. See Theorem 4.1.1 on pp. 21 of Hiriart-Urruty and Lemarechal [2D]. □ 
Proposition 6. Let e > and r € [0, 1]. If ip : [0, e) — )• R is convex, then I : (0, e) — )• 1R define by 

is increasing. 

Proof. See Theorem 4.1.1 and Remark 4.1.2 on pp. 21 of Hiriart-Urruty and Lemarechal |20j. □ 
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2 Local analysis for Gauss-Newton's method 

Our goal is to state and prove a local theorem for Gauss-Newton's method. First, we will prove 
some results regarding the scalar majorant function, which relaxes the Lipschitz condition of the 
derivative of the function associated with the nonlinear least square problem. Then we will show 
that Gauss-Newton's method is well-defined and converges. We will also prove the uniqueness of 
the solution in a suitable region and the convergence rate will be established. The statement of the 
theorem is as follows: 

Theorem 7. Let C R n be an open set, F : ft — > M. m a continuously differentiable function and 
m > n. Let x* £ ft, R > and 

c:= \\F(x*)\\, f3: = \\[F'(x*) T F f (xj\- 1 F , (x*) T \\, K:=sup{t£ [0,R) : B(x*,t) C ft} . 

Suppose that X % tS CI solution of (pQ) ; F'(x*) has full rank and there exists a f : [0, R) — > R 
continuously differentiable such that 

\\F\x) - F'{x* + t(x - x m ))\\ < f (||x - x*||) - /' (t\\x - x,\\) , (2) 

for all t 6 [0, 1], x £ B(x*, k) and 

hi) /(0) = and f (0) = -1; 

h2) /' is convex and strictly increasing; 

h3) V2c(3 2 D + f{0) < 1. 

Let be given the positive constants v := sup{t € [0, R) : /3[f'(t) + 1] < 1} , 

p := sup jt e (0, u) : t[l - /3(f(t) + 1)] < l ] ' r := mm{K ' "> ' 

Then, the Gauss-Newton's method for solving ([1]), mf/i starting point xo € B(x*, r)/{x*} 

x k+l = x k - [F'{x k ) T F'{x k )} _1 F'(x k ) T F(x k ), k = 0,1... , (3) 
is well defined, the generated sequence {x k } is contained in B(x*,r), converges to x* and 

H || < ^Lf(|ko - g*J|)Jko - g*Jj ~ /(Iko ~ ^*ll)] I, _ ,,2 

" Xfe+1 ||x -x*P[l-/3(/'(||x -x*||) + l)] X *" 

+ ||xo-x,||[l-/3(/'(||x -x,||) + l)] l|Xfe fe - u '^---- W 

Moreover, if [(3(pf'(p) - f{p)) + V2cp 2 (f'(p) + l)]/[p(l - p(f'(p) + 1))] = 1 and p < k, then r = p 
is the best possible convergence radius. 
Lf, additionally, 
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h4) 2 c/3q D + f'(0) < 1, then the point x* is the unique solution of (pQ) in B(x*,a), where 

0<a:= sup{t G (0, «) : &9(/(t)/< + 1) + cA,(/'(t) + 1)/*] < 1}, A) := ||[F'(x,) T F'(x,)]- 1 ||. 

Remark 1. T/ie inequality @ shows that if c = ff/ie so-called zero-residual case), then the 
Gauss-Newton's method is locally Q-quadratically convergent to x*. This behavior is quite similar 
to that of Newton's method (see ffj^ \3$). If c is small relative (the so-called small-residual case), 
the inequality (j4|) implies that the Gauss-Newton's method is locally Q-linearly convergent to x#. 
However, if c is large (the so-called large-residual case), the Gauss-Newton's method may not be 
locally convergent at all, see condition h3 and also example 10.2.4 on PP-225 of JIJ/. Hence, we may 
conclude that the Gauss-Newton's method perform better on zero-or small-residual problems than 
on large-residual problems, while the Newton's method is equally effective in all these cases. 

For the zero-residual problems, i.e., c = 0, the Theorem [7] becomes: 

Corollary 8. Let fi C R n be an open set, F : Q — > R m a continuously differ entiable function and 
m > n. Let x* G $7, R > and 

P := \\[F'(x*) T F\x*)\- x F\x*) T \\ , k := sup{t G [0,12) : B(x*,t) C 0} . 

Suppose that F(x*) = 0, F'(x*) has full rank and there exists a f : [0, R) — > R continuously 
differentiable such that 

\\F'(x) - F'(x* + t(x - x*))|| < f {\\x - x,\\) - f (r\\x - x*||) , 

for all t G [0,1], x £ B{x*, k) and 

hi) /(0) = and f (0) = -1; 

h2) /' is convex and strictly increasing. 

Let be given the positive constants v =: sup{i G [0, v) : /3[f'(t) + 1] < 1}, 

p := sup{t G (0, u) : [P(tf'(t) - /(*))]/[t(l - /3(/'(i) + 1))] < 1}, r := mm{ K , p} . 

Then, the Gauss-Newton's method for solving ([I]), with initial point xq G B(x*, r)/{x*} 

x k+1 = x k - [F'(x k ) T F'(x k )] ~ l F'(x k ) T F(x k ), k = 0,1..., 

is well defined, the sequence generated {x k } is contained in B(x*,r) and converges to x* which 
is the unique solution of (P) in B{x if ,a), where < a := sup{0 < t < n : f3[f(t)/t + 1] < 1}. 
Moreover, there holds 

H _ n < P[f'(\\xQ ~ X*\\)\\X Q - X*|| - /(||x - X*||)] _ 2 _ 

X *"" ||xo-x,P[l-/3(/'(||x -x,||) + l)] X *" ' 

If, additionally, [j3{pf'{p) — f(p))]/[p(l — j3{f'{p) + 1))] = 1 and p < k, then r = p is the best 
possible convergence radius. 
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Remark 2. When m = n, the Corollary^ is similar to the result on Newton's method for solving 
nonlinear equations F{x) = 0, which has been obtained by Ferreira ffjjj in Theorem 2.1. 



In order to prove Theorem[7]we need some results. From here on, we assume that all assumptions 
of Theorem [7] hold. 

2.1 The majorant function 

Our first goal is to show that the constant n associated with Q and the constants v, p and a 
associated with the majorant function / are positive. Also, we will prove some results related to 
the function /. 

We begin by noting that k > 0, because O is an open set and 6 Q. 
Proposition 9. The constant v is positive and and there holds 



P[f'(t) + !]<!, 



t e (o,i/). 



Proof. As /' is continuous in (0, R) and /'(0) 



1, it is easy to conclude that 




Thus, there exists a S > such that P(f'(i) + 1) < 1 for all t € (0,5). Hence, v > 0. 
Using b_2 and definition of v the last part of the proposition follows. 



□ 



Proposition 10. The following functions are increasing: 



[0, R) 3t ^l/[l-/3(f(t) + l)}; 



ii) 



(0, R)3t^[tf(t)-f(t)]/t 



2. 



iii) 



(0, R) 3 t ^ [f(t) + l}/t; 



iv) 



(0, R) B t ^ f(t)/t. 



As a consequence, are increasing the following functions 





Proof. The item i is immediate, because /' is strictly increasing in [0, R). 

For proving item ii, note that after some simple algebraic manipulations we have 
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So, applying Proposition [6] with f = <p and e = R the statement follows. 

For establishing item iii use h2, f'(0) = — 1 and Proposition [6] with f' = ip,e = R and r = 0. 

Assumption h2 implies that / is convex. As f(0) = 0, we have f(t)/t = [f(t) — f(0)]/[t — 0]. 
Hence, item iv follows by applying Proposition [6] with / = ip and and r = 0. 

For proving that the functions in the last part are increasing combine item i with ii for the first 
function and i with iii for the second function. □ 



Proposition 11. The constant p is positive and there holds 
(3[tf'(t)-f(t)} + V2cp 2 [f'(t) + l] 



< 1, 



i[l-/3(/'(i) + l)] 
Proof. First, using hi and some algebraic manipulation gives 

g\ m /(t)~/(0) 

PW(t) - /(*)] + V2cf5 2 [f'(t) + 1] P [ J[l) t-0 



Vie (0, p). 



fit) - AO) 
t-o 



i[l -/?(/' (t) + l)] l-/3(/'(t) + l) 

Combing last equation with the assumption that /' is convex, we obtain from Proposition [5] that 



WW 



f(t)] + V2c^[f'(t) + l] _ ~* D+ w 
. - 0(f(t) + 1)1 - V ' C/ " ^ ; 



!™ /[i - ;(/'(/> h- Di 



(0). 



Now, using h3, i.e., V2cp 2 D + f'(0) < 1, we conclude that there exists a 5 > such that 

-fit)} + V2c(3 2 [f(t) + l] 



< 1, 



t G (0,5), 



t[l-/3(/'(t) + l)] 
Hence, S < p, which prove the first statement. 

For concluding the proof, we use the definition of p, above inequality and last part of Proposi- 
tion Q3B □ 

Proposition 12. The constant a is positives and there holds 

/3(f(t)/t + 1) + cW(t) + l)/t < 1, t € (0, a). 
Proof. For proving that <r > we need the assumption h4. First, note that condition hi implies 



t 



+i 



fit) + 1 



p 



f(t) - 7(0) 

t-0 



/'(0) 



f (t) - f (0) 
t-0 



Therefore, using last equality together with the assumption that /' is convex and h4 we have 
limt_M)[/9(/(t)/t + 1) + cp Q (f(t) + l)/t] = c/3 0j D + /'(0) < 1/2. Thus, there exists a <5 > such that 

"/(*) 



/5 



+ 1 



i G (0,5). 



Hence, 5 < a, which prove the first statement. 

For concluding the proof, we use the definition of a, above inequality and items iii and iv in 
Proposition [TU1 □ 
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2.2 Relationship of the majorant function with the non-linear function 

In this section we will present the main relationships between the majorant function / and the 
function F associated with the nonlinear least square problem. 

Lemma 13. Let x G fi. If \\x — < min{z^, k}, then F' (x) T F' (x) is invertible and the following 
inequalities hold 

\\\F'(x) T F'(x)r 1 F'(x) T \\< 



i-p[f'(\\x-x4) + iy 

and 

^F'ixfF'ix^F'ixf - [F'(x*) T F'(x*)}- l F'(x*) T 
In particular, F'{x) T F'{x) is invertible in B(x*,r). 



l-P[f'(\\x-x4) + l]' 



Proof. Let i 6 SI such that \\x — x*\\ < min{i/, k}. Since ||x — x*\\ < using the definition of /3, 
the inequality (|2|) and last part of Proposition [9] we have 

\\F'{x) - F'ixJWWiF'^fF'ix^F'fafW < (3[f'(\\x - - /'(0)] < 1. 

For simply the notations define the following matrices 

A = F'(x*), B = F'(x), E = F'(x)-F'(x*). (5) 

The last definitions together with latter inequality imply that 

ii^t|i<raiAt||<i, 

which, using that F'(x*) has full rank, implies in view of Lemma [2] that F'(x) has full rank. So, 
F'(x) T F'(x) is invertible and by definition of r we obtain that F'(x) T F'(x) is invertible for all 
x G B(x*, r). 

We already knows that rankF'(x) = rankF' (x*) = n. Hence, for concluding the lemma, first 
use definitions in ([5]) to obtain that rank(B) = rank(A) = n and then combine the above inequality 
and Lemma [3j □ 

Now, it is convenient to study the linearization error of F at point in $7, for that we define 

E F (x,y):=F(y)-[F(x)+F'(x)(y-x)], y, x G fl. (6) 

We will bound this error by the error in the linearization on the majorant function / 

e f (t,u):=f(u)-[f(t) + f'(t)(u-t)], t,ue[0,R). (7) 

Lemma 14. If \\x — < k, then there holds \\Ef(x, < e/(||x — x*||, 0). 
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Proof. Since B(x*, k) is convex, we obtain that x* + t(x — x*) € B(x%, n), for < r < 1. Thus, as 
F is continuously differentiable in f2, definition of E F and some simple manipulations yield 



\\E F {x,x*)\\ < 



/ \\[F'(x) - F'(x* + t(x - x*))]\\ ||x* — x\\ dr. 
Jo 

From the last inequality and the assumption ([2D, we obtain 

\\E F (x,x*)\\ < [ [f'(\\x-x4)-f'(T\\x-x4)]\\x-x4dT. 
Jo 

Evaluating the above integral and using definition of e/, the statement follows. □ 

LemmaPT3l guarantees, in particular, that F'(x) T F'(x) is invertible in B(x*, r) and consequently, 
the Gauss-Newton iteration map is well-defined. Let us call Gp, the Gauss-Newton iteration map 
for F in that region: 

G F :B{x*,r) -> W 1 

x ' ^ x- [F'{x) T F'{x)]~ l F'{x) T F{x). U 

One can apply a single Gauss-Newton iteration on any x € B(x*,r) to obtain Gp{x) which may 
not belong to B(x*,r), or even may not belong to the domain of F. So, this is enough to guarantee 
well definedness of only one iteration. To ensure that Gauss-Newton iterations may be repeated 
indefinitely, we need following result. 

Lemma 15. Let x € O,. If \\x — x*|| < r, then G F is well defined and there holds 

" F{) ||x-x,f[l-/3(f(||x-x,||) + l)] l|X X * 11 

V2c^[f(\\x-x4) + l] „ I, 
+ ||s-x»||[l-/3(/'(||x-x,||) + l)] 11 

/n particular, 

\\G F (x) — x*|| < ||x — x*||. 

Proof. First note that, as ||x — x*|| < r it follows from Lemma [13] that F'{x) T F'{x) is invertible, 
then G F (x) is well defined. Since F'(x*) T F(x*) = 0, some algebraic manipulation and ([8]) yield 

G F (x) - x, = [F'(x) T F'(x)] ~ V(x) T [F'(x)(x - x m ) - F(x) + F(x*)] 

+ [F'(x*) T F'(x*)]~V(x*) T F(x*)- [F'(x) T F / (x)]" 1 F'(x) T F(x,). 
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From the last equation, properties of the norm and ([6]), we obtain 



G F (x)-x4 < \\[F'(x) T F'(x)]-'F'(x) T \\ \\E F (x,x 



+ 



[F'(xj r F'(x,)]- 1 F , (x,)' r - [F'{x) T F'{x)] ~ l F'(x) T \\F{x* 



Since c = ||-F(x*)||, combining last inequality with Lemmas 1131 and 1141 we have 



\G F (x) —x#\\ < 



(3e f (\\x - x*\\,0) 



+ 



V2c(3 2 (f'(\\x-x4) + l) 



l-P(f'(\\x-x4) + l) l-(3(f'(\\x-x4) + l) 
Now, using (J7D and hi, we conclude from last inequality that 



\\G F (x)-x4 < 



p\n\\x-x> 



\X X * 



f(\\x - X, 



l- Wdlx-x^lD + i) 



+ 



V2c/3 2 [f( 



\x x * 



+ 1] 



l-/3(/'(||x-x*||) + l)' 



which is equivalent to the first inequality of the lemma. 

To end the proof first note that the right hand side of the first inequality of the lemma is 
equivalent to 



P[f'(\\x - x4)\\x - x4 - f{\\x - X*\ 
||x-x,||[l-/3(/'(||x-x,||) + l)] 



+ 



V2cP 2 [f'(\\x-x4) + l} 
\x-x4[l-P(f'(]\x-x.\\) + l)] 



\X X ^ 



On the other hand, as x G B(x* : r)/{:c*}, i.e., 0< \\x — <r< p we apply the Proposition [TT] 
with t = \\x — x4 t° conclude that the quantity in the bracket above is less than one. So, the last 
inequality of the lemma follows. □ 

2.3 Optimal ball of convergence and uniqueness 

In this section, we will obtain the optimal convergence radius and the uniqueness of the solution. 

Lemma 16. // (P{pf'(p) - f(p)) + V2c/3 2 (f'(p) + l))/p(l - (3(f(p) + 1)) = 1 and p < k, then 
r = p is the best possible. 



Proof. Define the function h : ( 

h(t) = r 



) -> R by 

t/P + t-f(-t), t€(-K, 0], 

t/(3 + t + f(t), te[0,K). 



(9) 



It is straightforward to show that h(0) = 0, h'(0) = -1//3, h'(t) = -1//3 + 1 + f'(\t\) and that 
\h'(t)-h'(rt)\<f'(\t\)-f'(T\t\), re [0,1], t€(-K,«). 
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So, F = h satisfy all assumption of Theorem [7] with c = \h(0)\ = 0. Thus, as p < k, it suffices 
to show that the Gauss-Newton's method applied for solving (pQ), with F = h and starting point 
xq = p does not converges. Since c = our assumption becomes 

W(pf( P ) - - Kf'(p) + 1)) = i- (io) 

Hence the definition of h in ([9]) together with last equality yields 

h'(p) T h{p) _ -p/P + p + f(p) _ J P(pf'(p) - f(p)) \ _ 

1 p voodoo p " + p ' 

Again, definition of h in Q and assumption (fTUj) gives 

hH-pYK-p) p/P-p- Hp) ( W(j>) - Hp)) 



x 2 = -p 



h>{-pYh'{-p) ~ " -I/? + 1 + f'(p) " \p(l - /3(/'(p) + 1)) 



Therefore, Gauss-Newton's method, for solving (TjQ) with F = h and staring point xo = p, produces 
the cycle 

Xo = P, XI = -p, X2 = p, ■■■ , 

as a consequence, it does not converge. Therefore, the lemma is proved. □ 
Lemma 17. If additionally, h4 holds, then the point x* is the unique solution of ([T|) in B{x*,o~). 
Proof. Assume that y G B(x*, a), y / x* is also a solution of ([1]). Since F'{y) T F{y) = 0, we have 

j, - = y - x, - [F / (x,) T F'(x,)]- 1 F , (y) T F(y). 
Using F'(x*) T F(x*) = 0, after some algebraic manipulation the above equality becomes 

y - x* = [F'{x*) T F\x*)}~ l F\x*) T [F'{x*){y - x*) - + F(x*)] 

+ [F'{x*) T F'(x*)]- l {F\x*f - F\y) T )F{y). 

Combining the last equation with properties of the norm and definitions of c, f3 and /3q, we obtain 

\\y - a7*|| < /? / ||F'(x#) - F'(x* + u(y - x*))\\\\y - x*\\du + c(3 \\F'(x*) T - F'(y) T \\. 
Jo 



Using ([2]) with x = x* + u(y — x*) and r = in the first term of the right-hand side, and x = y and 
r = in the second term of the right-hand side in last inequality, we have 

\\y - x4 < (3 [\f'(u\\y - x*||) - f'(0))\\y - x*\\du + c/3 [f'(\\y - x*||) - /'(0)]. 
Jo 



11 



Evaluating the above integral and using hi, the latter inequality becomes 



\y - x 4 < P 



tt\\v-x*\ 
\\y - x *\\ 



+ 1 



+ cA> 



n\\y-x4) + l 
\\y - x*\\ 



\y 



Since < \\y — x*\\ < a, using Proposition [T2l with t = \\y — we have \\y — sc*[| < \\y — 
which is a contradiction. Therefore, y = x*. □ 

Remark 3. Note that in the above lemma we have used the fact that condition ([2]) holds only for 
t = 0. 



2.4 Proof of Theorem \7\ 

First of all, note that the equation in © together flSJ) imply that the sequence {xk} satisfies 

Xk+i = G F (x k ), k = 0,l,.... (11) 

Proof. Since xo G B(x*, r)/{x*}, i.e., < — < r, by combination of Lemma [T3l last 
inequality in Lemma [15] and induction argument it is easy to see that {x k } is well defined and 
remains in B(x*,r). 

Now, our goal is to show that {x k } converges to x*. As, {x k } is well defined and contained in 
B(x*,r), combining (111ft with Lemma[l5]we have 

H || ^ P[f'{\\ x k ~ x 4)\\ x k -X*\\- f(\\ x k ~ x *\\)\ . _ ||2 



x k -x4 2 [l- P(f'(\\x k -x4) + l)] 

V2c(3 2 [f'(\\x k -x4) + l] 



+ 



\x k -x4[l-(3(f'(\\x k -x4) + l)} 



I x k x% 



for all k = 0,1, Using again (jlip and the second part of and Lemma [TBI it easy to conclude 

that 



\%k ^ 1 1 ^ * 



k = 1,2. 



(12) 



Hence combining two last inequalities with last part of Proposition [10] we obtain that 

H I, ^ fiLTGko - g*j|)jko - - /(Iko - x 4)} n _ 1,2 



|x -x,|| 2 [l-/3(/'(||x -x,||) + l)] 



+ 



V2c/3 2 [/'(||*o-*,||) + l] 



|x -^||[l-/3(/'(||x -x*||) + l)] 
for all = 0, 1, . . ., which is the inequality Now, using (|12p and last inequality we have 

'P[f'(\\ x o ~ x*\\)\\ x o -x*\\- f(\\ x o - x 4)] + V2cP 2 [f'(\\x - x4) + 1] 



•£fC 1 1 J 



\x -x4[l-f3(f'(\\x -x4) + l)] 
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for all k = 0, 1, Applying Proposition [TTJ with t = \\xq — x*|| it is straightforward to conclude 

from latter inequality that {\\xk — x*||} converges to zero. So, {xk} converges to x*. The optimal 
convergence radius was proved in Lemma [16] and the last statement of theorem was proved in 
Lemma [T71 □ 

3 Special cases 

In this section, we present two special cases of Theorem [7J They include the classical convergence 
theorem on Gauss-Newton's method under Lipschitz condition and Smale's theorem on Gauss- 
Newton for analytical functions. 

3.1 Convergence result for Lipschitz condition 

In this section we show a correspondent theorem to Theorem [7] under Lipschitz condition (see pQ 
and [2] ) instead of the general assumption (|2|). 

Theorem 18. Let f2 C M. n be an open set, F : Q — >■ M m be continuously differentiable in Vt and 
m > n. Let x* G and 

c:= \\F(x*)\\, /?:= \\[F , (x m ) T F J (x m )]- 1 F'(x*) T \\, «:=sup{te [0,12) : B(x*,t) C 0} . 

Suppose that x* is a solution of (pQ) ; F'(x*) has full rank and there exists a K > such that 

V2cf3 2 K<l, \\F'(x)-F'(y)\\<K\\x-y\\, V x, y G B(x t , k). 

Let 

r : = min |k, 

Then, the Gauss-Newton methods for solving ([JJ), with initial point xo G B(x*, r)/{x*} 

x k+1 =x k - [F'{x k ) T F'{x k )] _1 F'(x k ) T F(x k ), k = 0,1, 

is well defined, the sequence generated {x k } is contained in B{x*,r), converges to x* and 
ii _ P K _ i|2 , yftcpK „ 

l|Xfc+1 X *" - 2(l-/3^||xo -x,||) l|Xfc X *H +1-/3^11x0 -x,|| l|Xfc fe - u '^---- 

Moreover, £/ (2 - 2V2K(3 2 c)/(3KP) < k, then r = (2 - 2V2K/3 2 c)/(3Kj3) is the best possible con- 
vergence radius. 

Lf, additionally, 2c/3qK < 1, then the point x* is the unique solution of (UJ) in B(x*, (2—2c/3oK)/(/3K)), 
where fa := \\[F*(x*) T F'{x*)]- 1 \\. 
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Proof. It is immediate to prove that F, x* and / : [0, k) ->• R defined by f(t) = Kt 2 /2 - t, satisfy 
the inequality ([2]), conditions hi and h2. Since y/2c/3 2 K < 1 and 2c(3 K < 1 the conditions h3 
and h4 also hold. In this case, it is easy to see that the constants v and p as defined in Theorem [71 
satisfy 

< p = (2 - 2V2Kf3 2 c)/(3K(3)) <v = 1//3K, 
as a consequence, < r = min{K, p}. Moreover, it is straightforward to show that 

W{p) - fip)) + V2c/3\f(p) + l)]/[p(l - p(f'{p) + 1))] = 1, 

and [/3(f{t)/t + 1) + c/9b(/'(t) + l)/t] < 1 for all t G (0, (2 - 2c/3oK)/(fiK)). Therefore, as F, r, f 
and satisfy all of the hypotheses of Theorem[7J taking xq G B(x*, r)\{x*} the statements of the 
theorem follow from Theorem [7J □ 

For the zero-residual problems, i.e., c = 0, the Theorem 1181 becomes: 

Corollary 19. Let O C 1" fc an open set, F : $7 — > M m be continuously differ -entiable in Q and 
m > n. Let x* € f2 and 

P ■= \\[F'(x*) T F'(x+)}- 1 F'(x*) T \\, « := sup{t G [0,i2) : B(x*,i) C fl} . 

Suppose that F(x*) = 0, F'(x*) has full rank and there exists a K > such that 

\\F'{x)-F\y)\\ < K\\x-y\\, V x, y G B(x*,k). 

Let 

r := min{K, 2/(3Kp)} . 
Then, the Gauss-Newton methods for solving (pQ), with initial point xq G B(x*, r)/{x*} 

x k +i = x k - [F'(x k ) T F'(x k )] ^ F'(x k ) T F(x k ), k = 0, 1, ... , 

is we// defined, the sequence generated {x k } is contained in £?(x*,r) and converges to x* which is 
the unique solution of (fl]) in B(x%,2/(/3K)). Moreover, there holds 

II _ ? K _ ,,2 ,_ nl 

llxfc+i S 2(i_^|| Xo X * 11 ' ^-U' 1 ---- 

//, additionally 2/(3Kf3) < k, then r = 2/(3K/3) is the best possible convergence radius. 

Remark 4. When m = n, the Corollary merge in the results on the Newton's method for sol- 
ving nonlinear equations F(x) = 0, which has been obtained by Ferreira |6j/ in Theorem 3.1 and 
Remark 3.3. 
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3.2 Convergence result under Smale's condition 

In this section we present a correspondent theorem to Theorem [7] under Smale's condition. For 
more details about Smale's condition see |16j . 

Theorem 20. Let £1 C R n be an open set, F : Q — > R m an analytic function and m > n. Let 
G Q and 

c := \\F(x*)\\, := ||[F'(a;*) T F'(a;*)]- 1 F'(a;») r || > « := sup{t > : B(x*,t) C O}. 
Suppose that solution of ([1]), F'(x*) has full rank and 

l/(n-l) 



7 := sup 

n>l 



n! 



< +oo, 



2\/2c/3 2 7 < 1. 



(13) 



Lei a := (2 + 3/3 - y/2cP 2 i), b := 4(1 + /3)(1 - 2^/20/327) and 

r := min {«, (a - Va 2 - &)/(2t(1 + £)) } • 
Then, the Gauss-Newton methods for solving (pQ), wii/i initial point xq G B(x*, ?")/{s*} 

x fe+1 = s fc - [F / (x fe ) T F / (x it )] _1 F'(x fc ) T F(x fe ), A; = 0, 1, ... , 
is we// defined, the sequence generated {xk} is contained in B(x*,r), converges to x* and 



+ 



(1 - 7||x - x*||) 2 - ^7(2 1| a?o - z*|| - l\\xo - x*\\ 2 ) Xk 
^/2c/3 2 7(2 -7||x -^||) 



(1 - 7||x - z*||) 2 - Pj(2\\x - x*|| - 7||x - x*\\ 2 ) 



2\ ll-^fc 3'* ||) k — 0, 1, . 



Moreover, if (a — y/a 2 — b) /{2 r y{\ + ft)) < k, then r = (a — \J a 2 — b) / (27(1 + /?)) is the best possible 
convergence radius. 

If additionally, 4c/3o7 < 1) ^en i/ie poin£ x* is £/ie unique solution ([1]) in B(x*,a), where a := 
(a;i- N /w?-^)/(27(l+^)), Wl := (2+/3-c/3 ), w 2 := 4(l+/3)(l-2c/3 7), ft := ||[F'(x*) T F'(x*)]||. 

We need the following result to prove the above theorem. 

Lemma 21. Let C R n 6e an open sei and F : — > R m an analytic function. Suppose that 
x* G $7 and B(x*, I/7) C fJ, where 7 is defined in (1130 . Then, for all x G B(x*, I/7) f/iere /10/ds 



|F"(x)|| < (2 7 )/(l-7||x-x t 
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Proof. Let x G £1. Since F is an analytic function, we have 

F"{x) = Y -F^ n+2 \x,){x - x,) n . 
z — ' n! 

n=0 

Combining (|13p and the above equation we obtain, after some simple calculus, that 

oo 

||^'(x)||<7E(n + 2)(n+l)(7||: 



|x -xJ|) n . 



71=0 

On the other hand, as B(x*, I/7) C we have 7||sg — x*\\ < 1. So, from Proposition 0] we conclude 

2 



(l-7||x-x*||) 3 _ f) 



5> + 2)(n + l)( 7 || 



37 3> * 



Combining the two above equations, we obtain the desired result. □ 

The next result gives a condition that is easier to check than condition ([2]), whenever the 
functions under consideration are twice continuously differentiable. 

Lemma 22. Let C 1" be an open set, x* G Q, and F : $7 — > W m be twice continuously on Q. If 
there exists a f : [0, R) — > M trace continuously differentiable such that 

\\F"(x)\\^f"(\\x-x4), (14) 

for all x G O suc/i £/ia£ ||x — x*|| < -R. T/ien F and / satisfy ([2]). 

Proof. Taking r G [0, 1] and x G fl, such that x* + t(x — x*) G £1 and ||x — x*|| < R, we obtain that 

\\[F'(x)-F'(x*+t(x-x*))]\\< J \\F"(x*+t(x - x*))\\\\x - x^dt. 
Now, as || x — x*|| < R and / satisfies ()14p . we obtain from the last inequality that 



[F'{x)-F'(x* + t{x-x,))] II <y / 



''■' J -"x — xj|)||x — xJIcft. 



Evaluating the latter integral, the statement follows. □ 
[Proof of Theorem 120] . Consider the real function / : [0, 1/7) — > M. defined by 

f(t) = — 2t. 
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It is straightforward to show that / is analytic and that 



/(0) = 0, f'(t) = 1/(1 - 7 tf 



/'(<)) = -1, /"(t) = (2 7 )/(l- 7 t) 3 , / n (0) = n! 7 



for n > 2. It follows from the last equalities that / satisfies hi and h2. Since 2 v / 2c/3 2 7 < 1 
and 4c/?o7 < 1 the conditions h3 and h4 also hold. Now, as f"(t) = (2j)/(l — 7i) 3 combining 
Lemmas [22l [21] we conclude that F and / satisfy (j2J) with i? = I/7. In this case, it is easy to see 
that the constants v and p as defined in Theorem [7J satisfy 



< p = ( a _ Va 2 - 6)/(2 7 (l + /?))< «/ = ((1 + /3) - ^(1 + /3))/( 7 (l + 0)) < 1 7 , 

and as a consequence, < r = min{/t, /?}. Moreover, it is not hard to see that 

[P(pf'(p) - f(p)) + V2c/3\f'(p) + l)]/[p(l - /3(/'(p) + 1))] = 1, 

and [f3(f(t)/t + 1) + c/3 (/'(*) + 1)/*] < 1 for all t G (0,cr). Therefore, as F, cr, / and a;* satisfy 
all hypothesis of Theorem [7J taking xq £ B(x*,r)\{x*}, the statements of the theorem follow from 
Theorem [7J □ 
For the zero-residual problems, i.e., c = 0, the Theorem 1201 becomes: 



Corollary 23. Let SI C W 1 be an open set, F : 9, 
x* G VL, and 

(3:= ||[F'(x*) T F'(x*)]-^F'(x*) T ||, 
Suppose that F(x*) = 0, F'(x*) has full rank and 

F( n \x, 



— > M. m an analytic function and m > n. Let 
k := sup{t > : B(x*,t) C O}. 



7 := sup 

n>l 



l/(n-l) 



< +OO. 



Let 



r := min {«, (2 + 3/3 - V/3(8 + 9/3)) /(2 7 (1 + /3)) } . 
Then, the Gauss-Newton methods for solving ([1]), with initial point xq € B(x*, r)/{x*} 

x k+1 = x k - [F'(x k ) T F'(x k )} _1 F'{x k ) T F(x k ), k = 0, 1, . . . , 

is we// defined, is contained in B(x*,r) and converges to which is the unique solution of ([1]) in 
B(x*, 1/( 7 (1 + /?)))■ Moreover, there holds 

/3 7 



< 



(1 - l\\xo 



/3 7 (2||x - 05* || - 7 ||x - 37* || 2 ) 



k = 0,1,.... 



Lf additionally, (2+3/3-^/3(8 + 9/3))/(2 7 (l+/3)) < «, thenr = ((2+3/3-^/3(8 + 9/3))/(2 7 (l+/3)) 
zs t/ie &es£ possible convergence radius. 

Remark 5. When m = n, the Corollary \23\ is similar to the results on the Newton's method for 
solving nonlinear equations F(x) = 0, which has been obtained by Ferreira J6j in Theorem 3.4- 
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